posted 11-11-2006 11:37 AM         

Lets move this to a different thread. Its a hugely important topic, and has as much to do with policy matters and what people do with test as it does with science and what test results are capable of meaning.
(I got the .pdf-s up so you can see the plots, if you are interested).

r
[rnelson]
------------------
Ray,

I’ll gladly kick this off, as I’m the guy who fueled the current brouhaha. Also, it seems unlikely to me that anyone in the Utah-DoDPI cabal will be quick to embrace the role of a significant “art” element in polygraphy. Now before anyone’s blood pressure starts climbing, please know that I used the word “cabal” with tongue I cheek. The reason: I have the feeling – and it’s just a feeling – that there are movements afoot to drive a stake (figuratively, of course) through the hearts of the Cleve Backster and Jim Matte. There are other opponents of the Utah system, perhaps, that are on a similar hit list, but I’ll cite those two because I’m a Backster grad and have studied independently with Matte.

Speaking of blood pressure, I get a little steamed when I keep seeing the same disinformation about Backster’s ZCT as represented in the Raskin/Honts chapter on the CQT in Kleiner’s book. Here’s one example:

“Backster severely reduced the amount of pre-test discussion of comparison question to simply telling the subject that certain other questions needed to be asked.”

Huh? You kiddin’ me? Are we talking about the same Cleve Backster? Nothing could be further from the truth. Matte corrects this -- and other fallacies in the Raskin/Honts chapter’s error-riddled section on the Backster ZCT -- in his 2000 supplement.

More recently, Barry C posted that administering a Matte Q-T was no different or more difficult than any other ZCT. If anyone really believes that, then, in my view, they don’t understand the critical set-up of the inside track.

So, I’ve broached just one aspect of the “art” component in polygraphy. What say the rest of you folks? Is there a place for art in polygraph, or are we all headed toward a sterile new world order driven by the number-crunchers of the Utah-DoDPI machine? (Again, tongue in cheek.)

Caution: In addition to more roadwork and doubling my intake of wheat germ, I dusted off the striking mitts and gave the heavy bag a sound pummeling -- so let’s get the discussion on!

My congrats to Ted and best wishes for the newest member of his family.

posted 11-11-2006 12:29 PM         

I tried adding to the Utah post but I think we filled it up...it won't take any new posts??? The good news is:

IT'S A BOY!
Jake Lee Mattsen 8lbs 7oz 22"
His dad is currently assigned to the FLETC in Glenco Georgia so we flew him home overnight at the last minute. He arrived in time to participate in the whole thing! What an event! Mom, baby and Dad are fine! Thanks for all the happy wishes and thoughts guys! You are the best!

posted 11-11-2006 01:19 PM            

It's two pages. You posted that three times, but I just thought you were excited.

Dan,

Don't forget, Charles Honts is a Backster graduate too. I spoke with a Backster grad recently who introduces (or then did anyhow) as "buffers." I've had people come to me and ask if Backster teaches a watered-down setting of CQs because that's what they observed during QC. I've heard the same of other grads too, so maybe it's just something some people tend to forget, but how is beyond me. Setting CQs is of utmost importance. If you mess that up, your test is a waste of time.

I've said this before, and I stole it from somebody somewhere, but we don't have to do much of anything to get the DI to go DI, but we've got to work to get the NDI to go NDI. That is where the art/science debate really gets its thrust.

We've got to standardize as much as we can, but there will always be some art involved when it comes to dealing with people.

Dan and all, to quote use the following commands (there on this site somewhere):

Anyhow, back to the topic. It's interesting that Jim doesn't address some of those issues until his supplement came out (and yes, I read through every painful page of that too). When I read that debate, I got the sense it was an afterthought for Jim. Maybe Backster has changed (I would hope anyhow) in the 30 years since Honts graduated.

We can all read the "critical" set-up of the Matte test. It's not that tough. Even if you screw the "delicate" presentation and don't get the answer you want, Matte tells you a directed lie to the inside track is not necessary. In other words, let him answer it truthfully and you'll be fine. That would seem to fly in the face of the reasoning behind his set up, but perhaps I'm being too logical.

Anyhow, since you insist I'm missing something, why don't you explain it here, so we can discuss something tangible? in other words, what do you do to set up the Matte QT ZCT? I'm sure many are curious.

(Matte's idea of a directed lie - something he hates, which makes this interesting - is different from that of those who actually use directed lies, another interesting topic.)

There's noting completely sterile about the Utah or DoDPI approach, and that's the problem for some, from the science angle anyhow. The DLCQ test remedies most of those problems, but the polygraph community has been slow to embrace that technique.

As an aside, have you read any of Dick Arther's stuff? In my opinion, a lot of what I see from Matte is repackaged Arther.

Finally, let me ask this foolish question:

What do you think of running a Bi-Zone (spot, I know) test in which the questions are just those of the inside track regarding the imminent "real" test? (I can't think of a better way to get people thinking. Sorry.)

R1 Are you hoping an error will be made on the test regarding the target issue?
R2 Regarding the target issue, are you hoping...?

C1 Are you afraid an error will be made on the test regarding the target issue?
C2 Regarding the target issue, are you afraid...?

What do you think the accuracy rate of that test would be? 100%? Less?

[This message has been edited by Barry C (edited 11-11-2006).]

[This message has been edited by Barry C (edited 11-11-2006).]

Okay Dan. I tried to tell you how to quote, but it kept seeing it as a quote, so it didn't work. Anyhow, here's the link to tell you how to do that and then some:
http://www.polygraphplace.com/ubb/NonCGI/ubbcode.html

posted 11-11-2006 03:39 PM            

quote:

---

... I have the feeling – and it’s just a feeling – that there are movements afoot to drive a stake (figuratively, of course) through the hearts of the Cleve Backster and Jim Matte.

---

Ahah!

A res gestae admission.

I knew it.

Cleve is a vampire.

How else could he look so young? (This was first suggested to me at APA, by a recent Backster grad.)

quote:

---

More recently, Barry C posted that administering a Matte Q-T was no different or more difficult than any other ZCT. If anyone really believes that, then, in my view, they don’t understand the critical set-up of the inside track.

So, I’ve broached just one aspect of the “art” component in polygraphy. What say the rest of you folks? Is there a place for art in polygraph, or are we all headed toward a sterile new world order driven by the number-crunchers of the Utah-DoDPI machine? (Again, tongue in cheek.)

---

Dan,

I think a lot of people would agree that the polygraph, and interrogation, are about half art and half science.

It seems, however, like we sometimes mistake one for the other.

The art part of polygraph has to do with getting the test done in a manner that is productive, efficient, ethically sound, recognizable, and reasonably consistent (given the considerable variability among test subjects, testing paradigms/goals, and investigation targets).

The science part has to do with prediction and classification – which have everything to do with measurement and predictive or inferential statistics. The goals in science are to achieve reliable measurements of features that are correlated with or predictive of our criterion (deception). Because all of the physiological features that we measure have multiple causes and correlated human activities, we are left to use the aggregated correlational efficiency of multiple distinct though related phenomena to support the accuracy of our diagnostic and classification schemes – and that means statistics, and that depends on measurements or observations that can be reproduced reliably. It is simply an accepted axiom in the sciences that reliability is achieved, among human raters, through structure and simplicity. The Backster/Matte and former DoDPI scoring schemes suffer a reliability disadvantage compared with the simpler Utah system, due to their complexity.

Of course, computers can achieve reliability even among complex schemes, because they do not fatigue or drift in their adherence to procedures. However, Backster/Matte, former DoDPI systems suffer another disadvantage with respect to computerized or mechanized scoring, because those systems depend upon tracing signature shape, or impressionistic quality instead of mechanical measurements.

Gordon & Cochette (1987) and Gordon (1999) offered a more mechanized form of tracing measurement in the Horizontal Scoring System, but it seems that system has receive little attention – perhaps because the authors did not describe how features were selected, but simply mechanized the measurement of features described in other scoring schemes. In contrast, Raskin and Kircher (1988, 2002) described their discriminate analysis that allowed for an empirically supported selection of salient phsyiological features that can be both mechanically measured and understood for their relative contribution to classification decisions. Similarly, the folks at JHU/APL have, though rather thinly, described their use of logistic regression to select physiological features for their relative contribution to classification decisions (the best description of this that I can find is in appendix F of the NAS/NRC report from 2003).

The Utah system offers distinct advantages to computer or mechanical measurement, compared with the Backster/Matte and former DoDPI systems, in that the specified features can be evaluated through mechanical measurements, which can then be subject to mathematical algorithms. Contrast that with the impressionistic signature features such as staircases, supressions, double-saddles, and the like – none of which are particularly well suited to obtaining mechanical measurements, and as such are evaluated based on the quality or shape of the tracing signature.

(Aside: I recently read a study that used Kolmogorov-Smirnov algorithms to authenticate human signatures – with accuracy rates in the .70s, excluding inconclusives.)

The Kircher features, employed by OSS and CPS offer additional advantages, in that simpler classification schemes seem to enjoy more success at cross-validation when compared with more complex schemes – possibly because of the tendency for more complex schemes to over-fit the development sample.

The fallacy of complexity can also be observed in testing and classification schemes outside polygraph. Look in Quinsey, Harris, Rice & Cormier (1998 I think) to find a study on risk assessments made by experienced forensic psychiatrists compared with the assessments made by high-school teachers. Investigators hypothesized that the experienced forensic psychiatrists would make more accurate predictions, would use more specialized methods of classification, and would look at different information – none of those hypotheses proved accurate. They found the risk classifications of high-school teachers (based on file information) to be just as accurate and more reliable than those of the experienced forensic psychiatrists. Moreover, they also observed that the experienced forensic psychiatrists were more confident in their classifications, than the high-school teachers, though with lower rates of inter-rater agreement (reliability).

So, experience and training, and all the LMNOP behind our names can make us more confident...

but that's not the same as accuracy.

Parallel that with the data that suggests that recent mental health graduates can sometimes do more accurate diagnostic work than experienced professionals... perhaps because they are more thorough and more cautious.

So, improving reliability, and the potential for validity, in polygraph depends in large part on reducing variability. That is perhaps nowhere more obvious, or more possible, than in the ways that we obtain and use scores. Numerical scores obtained through impressionistic (non-measurement) shape can be expected to always result in greater interrater variability than those obtained through mechanical procedures.

The point at which we swap the roles of art and science is when we begin to assume that thinks like “critical set-up of the inside track” are the most important answer to reliability challenges in polygraph. I am by no means suggesting that consistency in application of procedures is not important, and I think we can all imagine that adherence to good procedures will contribute to reliability. Its just that there will always be unmeasurable, and necessary variability in pretest administration – due to examinee/examiner expertise, prior testing experience, testing paradigm, and investigation targets.

As part of this “art,” we have to recognize that the way we pretest subjects at their first polygraph, may be necessarily different than the way we pretest them at their 30th or 50th polygraph – just as the way we pretest the neuropsychologist or physiology professor may be different than the way we pretest the transmission mechanic, Sheriff's deputy, or computer programmer. They all present different types of wisdom and intelligence. The goals in the pretest are to prepare and condition the examinee to complete the test. As Gordon Barland says (paraphrasing) “to encourage the truthful examinee to refrain from trying to alter the test, while encouraging the deceptive examinee to attempt to alter the test.”

(I heard from Gordon Barland, at the recent conference in Utah, that Raskin was trained at the Backster school.)

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-11-2006 04:40 PM         

I am not a polygraph researcher. I am a polygraph practioner. I only ended up in polygraph school because I pissed off my Chief one to many times!

I try to read at least an hour each day on polygraph.(When I'm done with the books I sell them on EBay to Barry). I try to attend at least two seminars each year to expose myself to other methods and new ideas.

I understand about half of what Barry posts and even less of what Ray posts! I do however, learn from each of them.

I am a Backster grad and apply the Backster method. On my DI criminal polygraphs, I have a confession/admission rate of better than 75%. Something about my training and the Backster method, is working, and working well. Keep in mind that since I work for the District Attorney, I am frequently prohibited from post test interrogation by defense counsel. I do however, enjoying seeing them entering a guilty plea after failing the exam.

What has been posted on this board is what is great about polygraph. It is also what is wrong with polygraph.

Disagreement between professionals opens the door to our critics but at the same time, it makes us better at what we do.

I guess my point here is that sooner or later, our profession must get our standards on the same page. That is why I joined ASTM.

I recently saw some charts where the examinee was told to think about the question for at least ten seconds before answering. Where the HAALE did that come from? It it valid? Is it reliable? Who invented it?

I also know of many law enforcement agencies who use a POT format for LEPET. Once again, where did that come from?

Differences of opinion are a good thing and only improve what we do. Standardization will have to be accomplished before we can ever expect to shut up the critics!

As far a Cleve and the vampire thing goes...those of you who know him know that he does stay up all night working in his lab.You make the call.......

Ted

[This message has been edited by Ted Todd (edited 11-11-2006).]

posted 11-11-2006 05:23 PM            

What is the ratio of DIs to NDIs in your tests? You have a confession rate of 75%, but what portion do you find DI?

Is the base rate of liars high? If you're doing them for the DA's Office, are they already charged by the time you get them?

Most of the tests I do for the DA's Office turn out to be NDI. But that's because a defense attorney doesn't agree to send a person to the police unless at least three factors are present: 1) they have no money for a private test (first anyhow), 2) the attorney senses something different about that particular client's denial, i.e., he's pretty confident the guy is actually telling the truth, and 3) the case isn't that great, but the risk is if they choose trial.

That is the opposite of what private examiners tell me, but then again, they're probably getting more test runs - tests I'll never see.

In those situations, I have never had a person go DI and not enter a guilty plea (to something), nor have I ever found a person NDI and not had the charges dropped (or not charged if the DA was already reviewing it).

I agree debate is a problem, but it's the only way to improve. What's nice about ASTM is that you've heard all these arguments before, haven't you?

posted 11-11-2006 06:44 PM         

I would say that about 60% of my tests run DI. Most are done before the suspect lawyers up. When the suspect has an attorney, and the attorney believes the client is innocent, they send them to me. In those cases, I would estimate that about 50% pass and we dismiss the charges. Frequently however, the suspect admits to some lesser involvement that helps lead to a new suspect.

In our county, the public defender has the money to pay for a private exam. Once their client passes the private exam, they will let me test the client on the same issue. The problem is, I have tested several of these guys after passing a private exam and them had them confess after my exam.

Defense attorneys seldom give private examiners all of the details of the crime prior to the exam. I have talked to private examiners after they passed a suspect that I failed. When I told the private examiner some of the hidden details, they have told me they wished they had that info prior to their test.

I again go back to my Backster training. I can't tell you how many times I have done a test on a "innocent" suspect who claimed no knowledge of the crime. I run either a Backster EXP or SKY and get admissions that there was indeed involvement.

You are soooo right that debate is healthy! If we had no critics, we would never improve. I have about a year left before I retire. Once I do, I intend to get more involved in research. I hope by then, that ASTM will be the collective clearing house and we can all be on the same page.

And for the record, I spent a few months with Cleve and he never bit my neck!

posted 11-11-2006 07:44 PM            

quote:

---

And for the record, I spent a few months with Cleve and he never bit my neck!

Ted

---

You see, that's the problem. Nobody ever remembers being bit.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-11-2006 09:19 PM         

Wish I had time tonight to address your points in detail, but I'm leaving bright and early tomorrow for PCSOT training at the Backster School and have 101 things to do beforehand.

Real quick...

Re: Set-up of the inside track
I find that every case is unique. First, I probe the examinee at length about his knowledge/attitudes/etc re the polygraph, then go from there. If fear of a FP is already present, it's easier; if no fear of a FP exists, then I gotta sell some fear, but buy enough of it back to get the "no" answer. In any case, the fear MUST be based on the examinee's lack of confidence in the process, NOT his innocence. I'll concede that it's tricky. The hope question is a no-brainer. (Example later.)

Re: Matte's "painful" supplement
I have no doubt that you were in pain when you read the unvarnished truth. :-) Are you saying that DLCQs do not contribute to FP results? Man, you Utah guys are hard core! Can you point to some field studies? College sophomores angling for beer money isn't all that convincing.

Re: "Inside-Track Only" Bi-Zone
Interesting approach. Too bad it can't be done with exlusionary CQs. But even if the inside track were satisfactorily shown to be rock solid, I doubt the feds would ever buy into it. Seems to me they'd rather avoid the tsunami of lawsuits that could result from those tested before the inside track was implemented. (DoDPI certainly has known about it.)

Re: "Ageless" Cleve
Amazing isn't it? Go figure.

Dan

posted 11-12-2006 01:00 PM            

Can someone email Dan a garlic talisman and and a kevlar turtleneck?

Hey, come to think of it you, and Matte, and Backster are all looking kind of "ageless."

------------------------

Seriously now.

I don't have Matte's 2000 supplement, and I've exhausted my budget on academic books for the year. (I'm thinking of simply leasing space and Barnes & Noble next year.)

Can someone explain the "painful" part?

I know there is a lot of controversy over DLCs, and that tends to be associated with the Utah techniques. However, I am of the impression that the majority of people using the Utah techniques are using PLCs.

What I find most “painful” is the impulsive tendency for examiners to indict each others' work, based not on the merits, empirical principles, or data, but on the “school” or personality of the leadership. I've heard of cases in which reviewers wouldn't support a test because the person didn't use a Backster test, didn't use a proper time bar, didn't wait 25 seconds, waited too long, asked about this or that, didn't ask about this or that, or some other issue that is not founded in data.

I think its important to understand that most of what we are all doing is probably quite similar. The details of test administration are critical, and it is very important for us to be willing to discuss our perceptions, assumptions, and techniques, and modify them based on emerging data. Inability or unwillingness to change would have us all driving cars with carburetors.

The measurement of objectivity is this: do we have the intellectual freedom and courage to prove ourselves wrong, discard ideas that cannot be supported and move toward ideas that are supported by data.

I don't think anyone perceives the DLC as a magic bullet thats going to solve all our problems without creating new ones. However, it does seem to suggest certain advantages. I haven't seen any thing recent on the TES, but I believe it is still a DLC test.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-12-2006 04:49 PM            

quote:

---

Re: Matte's "painful" supplement
I have no doubt that you were in pain when you read the unvarnished truth. :-) Are you saying that DLCQs do not contribute to FP results? Man, you Utah guys are hard core! Can you point to some field studies? College sophomores angling for beer money isn't all that convincing.

---

Unvarnished truth? Surely you jest? That was an attempt at humor I hope.

In the only field study I know of, the DLCQ test reduced - yes reduced - FPs from 20% or 0%. As far as lab studies go, are you saying they don't generalize to the field? So far that doesn't appear to be the case, and for info on that, see the CM publication I posted about yesterday as it discusses that issue. If I remember correctly, the NAS found a median accuracy of 85% and 86% for lab and field studies, so that appears to be a dead issue.

What is "painful" about Matte's writing is first and foremost his poor writing style. He needs to expand his vocabulary. I don't know how many times you will find the word "aforementioned" on any given page he writes, and I'm not the first to have pointed that out. Moreover, he introduces words and phrases, such as "forensic psychophysiologist," and almost every time adds, parenthetically, "FP." Just tell me your abbreviation and use it; otherwise you're just slowing me down! Additionally, he doesn't use science to back up his claims; rather, he resorts to what Ray makes mention of above.

Take a look at the DLCQ debate he and Honts got into back a ways in POLYGRAPH. It would be almost comical how Honts filleted him - were it not so sad we are having such rudimentary tiffs about basic science.

One of the reasons I am apprehensive about using any Matte test is for the reasons Ray cites. He thinks - wrongly - that any deviation from his "validity study" renders a test invalid. Forget the fact that there is no evidence to show any CQ question type (barred, exclusionary, non-exclusionary, etc) is better than another, he's sticking to his guns. Should I decide to run a Matte test without the black zone, he'd argue it's invalid. I don't need that headache.

(Why would I eliminate the black zone? All the evidence shows it does nothing to help the examinee, but there is one study (Honts and Amato) to show it does reduce the scores of the truthful (making a false positive more likely).)

quote:

---

Re: "Inside-Track Only" Bi-Zone
Interesting approach. Too bad it can't be done with exclusionary CQs. But even if the inside track were satisfactorily shown to be rock solid, I doubt the feds would ever buy into it. Seems to me they'd rather avoid the tsunami of lawsuits that could result from those tested before the inside track was implemented. (DoDPI certainly has known about it.)

---

So what you're saying is DoDPI knows they're wrong, but they are choosing to keep doing wrong because if they had the epiphany you had and changed their ways, it would be an admission of fault, opening the floodgates of lawsuits for the countless victims of their schemes? Apparently there's not a single ethical man in the bunch?

It can be done with exclusionary CQs - except in the minds of those who still think it matters, which goes back to Rays point.

quote:

---

Re: Set-up of the inside track
I find that every case is unique. First, I probe the examinee at length about his knowledge/attitudes/etc re the polygraph, then go from there. If fear of a FP is already present, it's easier; if no fear of a FP exists, then I gotta sell some fear, but buy enough of it back to get the "no" answer. In any case, the fear MUST be based on the examinee's lack of confidence in the process, NOT his innocence. I'll concede that it's tricky. The hope question is a no-brainer. (Example later.)

---

According to Matte, you don't need to get a "no" answer for the test to work. The question will cause enough arousal with a "yes" he says. In any event, how is that so much different than getting any other CQ set? I often use a hurt / harm CQ. When I first ask, "Have you ever hurt anyone?" - after my intro, I almost always get a "no." Then I go into my explanation of what "hurt" means to me: physical, emotional, financial - with examples, etc. And then... I watch him squirm. Once I am comfortable the CQ is set, I tell the person how glad I am to hear he hasn't done any of those things.... Some will be forced to confess something, which we all know. In the end, you're not doing anything any different than we are. I can sell the CQs as hard as I want, but you're saying you can't do that as you could lose your shot at a "no" answer. I would expect that to reduce the impact of the question, thus increasing the chances of a false positive. (I am of the school that you can't oversell a CQ, and one of the reasons I like the Utah test is that you stim them between charts - a big no-no in Matte's book, but a big yes if you believe in science over subjective opinion, intuition, divination, or whatever else one might use to arrive at an opinion or conclusion.

Look, I'm an investigator. I go where the evidence leads me. I joke with people that I don't have enough faith to be an atheist, but that's true, I don't. Can I err? Sure. Show me the evidence, and I can be persuaded. Right now, I have seen no credible evidence that the Matte test is any better than any other CQT - PLCQ or DLCQ. We don't even know if the fear of error question actually measures anything to do with fear as fear isn't even a necessary element in a CQT.

posted 11-14-2006 10:38 PM         

Real quick...

>>>Apparently there's not a single ethical man in the bunch?

Wasn't referring to DoDPI -- I'm talking about upper-echelon policy wonks at DOJ.

>>>Look, I'm an investigator. I go where the evidence leads me.

Congratulations. I'm sure the taxpayers are thrilled. But what about your instincts? Ever heed them? Or is there never a need in the face of "evidence"?

posted 11-15-2006 09:00 AM            

quote:

---

Congratulations. I'm sure the taxpayers are thrilled. But what about your instincts? Ever heed them? Or is there never a need in the face of "evidence"?

---

Instincts will betray us, so no, I don't put much faith in them. They're kind of hard to defend under cross-examination, and I don't make a practice of looking like a fool. I do that enough without trying. Sure though, there are times when a good investigator must use "instincts." Often that is just a way of saying, "I see something that I can't articulate, but we should go in this direction." We then look for evidence to corroborate it. If there isn't any, then we abandon our instincts.

I don't really think there is a lot of room for "instincts" in polygraph. We know that the theory behind polygraph is that one question type is more salient for the truthful, and one for the deceptive (to the RQs). Out job (in a PLCQ test) is to properly set CQs to give the innocent a place to dump his energy. It bothers me that some use "instinct" to know when that occurs, i.e., that the person is lying or unsure of his answer. There is enough science out there to give us a better basis on which to know that, but I've heard examiners base their decisions on nose scratches, eye movements, and a host of other unscientific means. (And we also know most - including polygraph examiners - are no better than chance at detecting deception, without the instrument, that is. Could it be that they look at the wrong (instinctual) cues?) We forget the old adage: garbage in, garbage out.

posted 11-15-2006 11:13 PM            

As Barry pointed out, instinct will provide you no better incremental validity than that already obtained through ones human ability to detect truth from lies. Barry also has pointed out that instincts are not better than the methods used in polygraph in providing incremental validity (although the research I have does show better than chance accuracy for some).

If you would like, you can read the research I speak of at the following links;
http://www.paulekman.com/pdfs/a_few_can_catch.pdf http://www.paulekman.com/pdfs/who_can_catch_a_liar.pdf http://www.paulekman.com/pdfs/ability_to_detect_deceit.pdf http://www.paulekman.com/pdfs/why_do_not_we_catch_liars.pdf

The bottom line is that the accuracy of human ability to discern truth tellers from liars ranged from 40-80 %, with the detection of truthful lower for most groups. Some groups had better accuracy in the studies but there still is no substantial evidence that the level of training will increase ones ability to be a human lie detector. These studies are not without faults and there are possibly other means that might boost accuracy that were not utilized or measured for.

Debates should for the most part focus on the facts surrounding an issue and not ad hominem attacks or other illogical arguments. For a reference see http://www.infidels.org/library/modern/mathew/logic.html .

Flippant remarks do nothing to further educated discussions of an issue or idea. Once I think that I know everything and the one way, I will presume I am dead.

I do not point this out to be critical but to aid us all in better debate (if or when such a case arises we might need to use these tools amongst those not our colleagues).

posted 11-20-2006 01:48 PM         

Re: Flippancy, etc. [J.B.Mc]
Barry C and I know each other casually, having met at a couple of events (APA/AAPP) in Maine. We had a few spirited discussions during AAPP, perhaps most notably concerning fidelity testing. Barry feels that absent an independent investigation, a polygraph test on fidelity issues is on thin ice. I disagree. If a couple stipulates that the key question is sexual relations outside of the marriage during a certain time frame, then what bearing does an investigation have? I mention this because in face-to-face discourse, the tone of discussion is evident. But in the one-dimensional and sometimes terse world of on-line communications, tone is often misconstrued. My original post in this thread makes reference to boxing metaphors such as “heavy bag” and “road work.” Thus, in my mind, I had set the stage for a somewhat breezy discussion. I’m not the kind of on-line writer who peppers his postings with smiley faces and emoticons. (Maybe I should.) At any rate, I have nothing but respect for Barry and the other experienced examiners who have taken time to help enlighten me. Barry, if I stepped on your toes, I am sorry and do indeed apologize.

Re: Instinct
I never meant to imply that a polygraph call should be based on instinct – hell, I’m a “charts only” Backster-taught examiner! The instinct angle I brought up was meant to embrace, in an ancillary way, developing one's lines of questioning, use of complex scoring rules, and the intuitive appeal of the “inside track” of Matte’s Quadri-Track system.

Re: “[DoDPI] polygraph practices…driven by politics or concern over litigation” [Don K]

Civilian mistrust of the government hovers at levels I haven’t seen since Watergate and Vietnam. Blame it on Iraq/WMD, congressional corruption, blood-for-oil, see-saw gasoline prices, outrageous corporate profits, take your pick… In my young polygraph career (two years), there have been three occasions where a prosecutor wanted to poly an attorney’s client. I told the lawyer that would be OK, but only if comprehensive quality-control measures were in place prior to the police exam. The universal response from the government was, “We’ll get back to you on that.” They never did. That’s not exactly a confidence-builder, in my view.

Anyway, the DoDPI “litigation sensitivity” issue is an unsavory scenario that’s convincingly depicted in a new book by Matte, “Truth and Deception.” It is the second part of his latest work “The Caul, A Trilogy.” Of course, it’s fiction -- more or less. I use the qualifier “more or less” because the novel contains many elements that are clearly non-fiction. (Go to Amazon and sample some pages in book 2, you'll see what I mean.)In any event, as Matte’s latest work becomes more widely read, such speculation on politics and litigation vis-à-vis polygraphy intensifies. And as Jim moves closer toward inking the deal for the movie rights, we should take pause. Now, just for the sake of discussion, what if Matte sold the rights to Michael Moore? Moore promptly sends out his bloodhounds, and lo and behold, they find one guy – just ONE guy – with a half-ass connection to DoDPI who states: “I have no doubt that DoDPI purposely ignored Matte’s Quadri-Track system, most likely because the government doesn’t want anything to do with a mechanism that verifies false-positive results.” Friends, that is a quotable quote that Moore would beat like a drum. I’d say that Maschke would do the same, but that might mean he’d have to admit that polygraph works, something he’ll never do. Still, George might find a way to get some mileage out of it, and the anti crowd would be off and running with another piece of propaganda… All of this begs the following question: If Matte’s Quadri-Track ZCT were satisfactorily validated – with accuracy rates equal to or better than Utah, with the added benefit of diagnosing false positive/negatives -- would DoDPI unconditionally embrace Matte’s system, incorporate it into its curriculum and advocate its use?

Re: “One of the purposes of the scientific method is to give us a means to test our instincts.” [Barry C]

I totally agree. To that end, I now seek an APA-approved seminar or workshop on the Utah system. Continuing education and polygraphy go hand in hand. In my case, I’m a Backster-taught examiner who adopted the Matte Quadri-Track system because I feel it’s better than Cleve’s standard You-Phase. I always take my time with exams (3-4 hours easy), run the Quadri-Track format by the book, and score somewhat conservatively. For now, the Matte Q-T ZCT fits my needs, and I’m very comfortable with it. But I’m open minded too. I realize that ASTM is the wave of the future, so I want to get formally educated on Utah.

Speaking of ASTM, Brother Sackett gave an impassioned speech on the merits of ASTM during training at Backster last week. Good work, Jim.

To reiterate, I’ve learned a lot through this forum and hope to learn much more. Meanwhile, I’d be grateful if anyone would recommend a highly regarded instructor who conducts an APA-approved seminar or workshop on the Utah system.

posted 11-20-2006 03:11 PM            

My name is Shawn Edwards. I am the Product Manager for Polygraph at Stoelting. I am new to the board, and just wanted to say that I really enjoy the lively and informative discussions here.

FYI: Dr. Kircher is the gentleman who is presenting the Utah Method at AAPP next year.

Happy Turkey Day to All!

posted 11-20-2006 10:22 PM            

quote:

---

Barry C and I know each other casually, having met at a couple of events (APA/AAPP) in Maine. We had a few spirited discussions during AAPP, perhaps most notably concerning fidelity testing. Barry feels that absent an independent investigation, a polygraph test on fidelity issues is on thin ice. I disagree. If a couple stipulates that the key question is sexual relations outside of the marriage during a certain time frame, then what bearing does an investigation have?

---

My feelings are ambiguous, and I understand both sides. However, I think a good investigation is essential to help prevent false positives - your main concern, and rightfully so.

This is the from the AAPP's Standards and Principles of Practice:

quote:

---

A. The polygraph examination should be a supplement to, not a substitute for, a field investigation.

B. The effectiveness of the polygraph examination, to a large extent, will be based upon the thoroughness of the investigation, prior to having the person take the examination.

C. To maximize the effectiveness of the polygraph examination, the investigator and the polygraphist must work together as a team.

---

The APA says the following in its Model Policy for Pre-employment Screening exams:

quote:

---

3.12.1.2 As with any polygraph examination, law enforcement pre-employment polygraph examinations do not take the place of an investigation.

---

How do you do the investigation and then act as the neutral polygraph examiner? If you are a good investigator, you're going to have to ask some hard questions, and they could be perceived as disbelief on your part - something you must avoid at all costs - unless you don't mind false positives.

I don't run tests on everybody I investigate. Instead, I ask tough questions - and often get confessions. I suppose I could skip that (investigative questioning) and go straight to the polygraph stage, but that seems to me to be a waste of resources and an improper use of polygraph.

If that's the case in a routine criminal investigation, why is it different in a fidelity test? I'm not ready to indict those who do such tests, but I think it raises a good ethical dilemma and debate.

When can we justify running tests without a thorough investigation? Is that something that will benefit the polygraph community and the people whom we serve?

posted 11-21-2006 12:45 AM            

As I stated at the end of my post to you, I was not trying to berate you or anyone else’s method of posting. My point was that, at least in a debate over serious issues, we should stick to a format of discourse that is consistent with how we would do so when representing our profession in public. This is a good forum to practice such debate and could serve us all well, if and when we might face anti-polygraph debate in a public forum.

That said, I would like to comment on some of the things you mentioned, from my perspective.

quote:

---

The instinct angle I brought up was meant to embrace, in an ancillary way, developing one's lines of questioning, use of complex scoring rules, and the intuitive appeal of the “inside track” of Matte’s Quadri-Track system.

---

This would be along the lines of a global approach and not “charts only”. Again, this will take you as far as your training and natural ability allows.

Barry and I discussed this very issue. The only way you will truly know what your ability to accurately do this is by testing it (e.g. blind chart scoring/evaluation, or using some other method to come to your decision against ground truth).

Just because the instinct is used subordinately in the method to diagnose deception and truth does not mean that it could not taint the entire diagnosis if improperly applied or inadequately done. This brings us back to ones ability to do this. It is not intrinsic in everyone and in some it is just not there.

If you look at the studies I provided, it is quite apparent where the increase in ability is found. Specific training can increase this ability (how much is not known) and those chosen for their ability did better than those not. Arguably the presence of both variables (training and ability) creates the best.

quote:

---

In my young polygraph career (two years), there have been three occasions where a prosecutor wanted to poly an attorney’s client. I told the lawyer that would be OK, but only if comprehensive quality-control measures were in place prior to the police exam. The universal response from the government was, “We’ll get back to you on that.” They never did. That’s not exactly a confidence-builder, in my view.

---

After I had completed two years of training and conducted over 200 criminal specific issue tests, I was finally allowed to proceed with the remainder of the requirements for full licensing in my state. I do not say this as an argument of authority nor to boast my qualifications (as this is relatively meaningless in the grand scheme) but to point out that qualifications for examiners are different in different arenas. And qualifications to review examinations are quite different where I conduct examinations. In my opinion, the only way one would know whether an individual could adequately conduct quality control is to test the individual’s ability (accuracy and competency).

Who would you suggest to quality control the exams you spoke of?

Just to set your mind at ease, every examination I conduct is subjected to quality control.

From my experience, when examinations are conducted for evidentiary purposes, outside quality control is a given, as both sides have the opportunity to evaluate the other's evidence independently. When the examination’s only probative value is for investigative purposes, quality control is then a private issue, as it is not evidence used in court and arguably not discoverable (FOI exception of investigative methods).

Quite often polygraph is held to a double standard. It is held up as a banner of truth for those who have favorable results but pooh-poohed when results are not to another’s satisfaction. Courts cringe when asked to admit it as evidence but often call for it on sentencing issues (maybe to help them feel warm and fuzzy with their decision).

The court case posted on Anti-polygraph.org is an example of this issue.

It is laughable to me when I see some of the arguments often used for the exclusion of polygraph results as evidence, as the same arguments ring true for every other forensic evidence admitted. What does the field research say about the accuracy of DNA? If DNA doesn’t have the potential of being given too much weight as evidence and in turn prejudicing the jury, I don’t know what does (the innocence project has hung their hat on DNA).

posted 11-21-2006 03:12 AM            

J.B. Wrote,

quote:

---

As I stated at the end of my post to you, I was not trying to berate you or anyone else’s method of posting. My point was that, at least in a debate over serious issues, we should stick to a format of discourse that is consistent with how we would do so when representing our profession in public. This is a good forum to practice such debate and could serve us all well, if and when we might face anti-polygraph debate in a public forum.

---

As form of guiding principle, I think it makes sense to strive for self-expression - speaking only for oneself, and referring to statements or publications to clarify the assertions of others. To make assertions for others without asking questions is what therapists call “mind-reading” and tends to fog the discussion. I find the most instructive conversations don't hinge heavily on rhetorical appeals to emotion but emphasize the credibility of assertions based on the authority of evidence, not simply the personality or personhood of asserter. Consider this, Ghandi was a reasonably smart guy, but he was quite wrong about hitler because he neglected to consider the evidence alongside his ethics and emotion regarding conflict.

“Evidenced based” is becoming a buzz-word in sex offender treatment circles.

Dan wrote,

quote:

---

Barry C and I know each other casually, having met at a couple of events (APA/AAPP) in Maine. We had a few spirited discussions during AAPP, perhaps most notably concerning fidelity testing. Barry feels that absent an independent investigation, a polygraph test on fidelity issues is on thin ice. I disagree. If a couple stipulates that the key question is sexual relations outside of the marriage during a certain time frame, then what bearing does an investigation have? I mention this because in face-to-face discourse, the tone of discussion is evident. But in the one-dimensional and sometimes terse world of on-line communications, tone is often misconstrued. My original post in this thread makes reference to boxing metaphors such as “heavy bag” and “road work.” Thus, in my mind, I had set the stage for a somewhat breezy discussion. I’m not the kind of on-line writer who peppers his postings with smiley faces and emoticons. (Maybe I should.) At any rate, I have nothing but respect for Barry and the other experienced examiners who have taken time to help enlighten me. Barry, if I stepped on your toes, I am sorry and do indeed apologize.

---

I too enjoy the debate, discussion and information on this forum. As a whole, I'm very impressed with the way these discussions have self-regulated without the need for moderation.

---------

I have a serious problem with fidelity tests, from a professional/ethical perspective. In short, professional assistance with solving relationship/marital difficulties (including trust and fidelity) is called "counseling" and that is a regulated activity in most, if not all, states.

Now we all know that therapists are like farts - some really stink, and some can provide a bit of relief - but it is important that we as polygraph examiners remain clear about the ethical and regulatory boundaries of our work.

To become involved as an investigator is certainly possible, and this makes most sense when the referral comes from a proper professional - a therapist (or I suppose, and attorney). The results of the investigation are then provided to the professional, who can engage any necessary follow-up discussion or action - keeping us polygraph examiners out of the marital and family therapy business. To do these tests from a self-referral or spousal referral is to invite all kinds of problems...

Ever read a polygraph report that reads "On such-and-such-date, I administered a polygraph to you..."

This is not very impressive. Imagine a police applicant, who shows up and says "I've already done my own background investigation," or "I've already taken my own polygraph test..."

Accepting referrals from other professionals stabilizes these problems to a degree. In medicine (perhaps not including managed-care situations and plastic surgery) specialists will generally want to see a referral from another professional - a generalist, who will sometimes follow up on the work of a specialist in a integrative manner.

Now imagine a woman calls to schedule a polygraph test to save her marriage to an abusive and suspicious husband (sorry guys, but most domestic violence and sexual offenders are men). Keep in mind that DV offenders are among our most lethal offenders, though they often receive only misdemeanor convictions and short sentences - with a maximum of 36 weeks of finger-tapping DV "classes." Most DV offenders can hold their breath for 36 weeks - we should look at post-conviction DV testing. Anyway, this female subject is going to be in trouble whether she passes or fails the polygraph. If she fails - his suspicions are proved and his rage is justified. If she passed, then he was WRONG, and that's a narcissistic injury to a DV offender. (Hint: narcissistic injuries, to narcissistic offenders, are reaction-producing injuries - and I don't mean polygraph reactions.) This couple needs professional guidance beyond a polygraph.

Now another scenario - a husband, cheating or not, schedules a polygraph to appease his wife's concerns. In this case, with no referring professional, if he fails the wife will never learn of the polygraph. And if he passes, the wife's concerns are trivialized (and perhaps ridiculed or worse). This couple again needs professional guidance to determine how to have a more effective relationship, or whether the relationship is even viable.

This gets more complicated when there are children involved.

It became painfully clear to me that fidelity testing is an unregulated practice, meddling in a regulated profession, when I observed as an intern, while another examiner conducted a fidelity test several years ago. The examiner is a good and well respected professional, and a former therapist. The female subject (age young 20s) had a jealous husband who was convinced she was seeing others. She was a stay-at-home mother with three children - ages 1, 2, and 3. (Not sure when she would have time for an affair.) She did not work, but her husband paid the $450 dollars for a polygraph test, which was conducted with the three children in the room during the examination. The two-year-old was nestled in a child-carrier/car-set thing, while the subject held the sleeping infant in her arms during the polygraph test. The three-year-old was awake and wandering about the room during the test. Part-way through the "test" the two-year old awoke and began to cry, so the mother rocked the child-seat with her foot for the remainder of the "examination." As an unregulated test, there was no requirement for video/audio recording.

Despite those distractions, the examiner scored the test to a non-deceptive conclusion and produced a report. Within the next week or two the husband was contacting the polygraph office, upset because he was proved wrong, though he was sure she had done something to beat the test. The woman also contacted the office asking the examiner to talk to her husband, who remained upset. The couple separated some months later.

This couple's problems are not going to be solved by a polygraph test. Since observing that test, I have steadfastly counseled fidelity test inquiries to counseling, including making several referrals to therapist whom I know to be competent.

While I believe in marriage and therapy, I also know that some marriages are better/safer once they are ended. Now it could be said that the involvement of the polygraph helped clarify the situation for the parties involved, and that conversations with the polygraph examiner may have helped people. But counseling is a regulated profession - for good reason.

If the polygraph profession has something to offer, in terms of access to information, deterrence, or decision-support value to offer marital situations, then we should regulate such professional involvement through our standards of practice - in that way we can become involved with a thoughtful understanding of the ethics and complications associated with our work.

I conduct fidelity/relationship tests only when they are referred to me by a therapist, and that is the only person who will get the results. The last one I did was a year or more ago - on a former priest (though that may be questionable, as he was from Francis Schuckhardt's schismatic catholic sect), about to get married to a massage therapist, though he was recently arrested for soliciting a prostitute. In his conversation with me, he revealed that he had sexually touched the teenage daughter of a family from his parish, before he was asked to leave the priesthood following his earlier prostitution arrests.

I'm sure not every test is as dramatic or clear as that, but for me it illustrates the problem - fidelity testing is undefined and unregulated. Under those circumstances, almost anything goes.

-----------------------

Now aside from all that seriousness, I hope nobody is offended by my attempts at humor and the "ageless" thing. I was going to suggest that someone check on Dan's nocturnal activities since returning, but I just realized its midnight and I should probably call it quits.

This has been a tough week that saw the demise of my favorite motorcycle - to a drunk to scattered it about the street and ditch in front of my home. I know better than to leave a bike on the street, but just one night and I'll ride it in the morning... "hey look at that neat old motorcycle (clang)". 26 years and 86,000 miles, only to end while standing still.

So now the tough question. do I put the basket-case back together, or look for an old Porsche as my next project? Looking at that bike is enough to make me never want to be involved in a bike accident.

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-21-2006 11:57 AM            

I hope that what I wrote was not misconstrued to be that I was speaking for others nor spreading “rhetorical appeals to emotion”. My assertions were not meant to do this but to illustrate my own opinion of how I believe things should be done based on my own personal experiences (e.g. discussions and lengthy public debates with anti-polygraph individuals).

I believe that certain areas are not truly scientific but are more commonsensical in nature. I have met many intelligent individuals who research complicated law issues with ease but cannot “walk and chew gum” at the same time.

Experience has shown me that I will perform as I practice. This has been even more the case when I find myself in a stressful environment.

posted 11-21-2006 12:26 PM         

>>>Looking at that bike is enough to make me never want to be involved in a bike accident.

WHAT? You're heeding your instincts over the precious DOT "data"? Say it ain't so, Ray, say it ain't so!

The "data" says you'll probably be OK. The "data" says that boomers just entering the sport -- or those re-entering after a long absence -- are the ones most at risk. That isn't you. You wear a helmet, right? You're a MSF grad, right? You have many years and many thousands of miles experience, right? No worries, mate. Just trust the numbers. Remember, there's only a one-in-ten chance that S&W .500 will go BANG. Hey, the "data" *IS* good enough for you, isn't it?

OK, I'm yankin' yer chain, bro', but I do feel your pain. I've been riding since 1970. My first bike was a '66 H-D 'CH (magneto ignition). I couldn't count the bikes I've owned, and I've ridden evrthing from SuperCubs to a Boss Hoss. But two years ago I decided to quit while I was ahead. I, too, have been tinkering with sports cars, but it ain't quite the same. So, as a compromise, I rent a Dyna-Glide for a few days once or twice a year, just to get it out of my system. Meanwhle, I have a lot of cheap fun with my Honda del Sol.

Instincts. Can't live with 'em, can't live without 'em. :-)

posted 11-21-2006 03:15 PM            

Not at all, I was reacting more to Dan's statement about "Barry thinks..."

We just finished an election in which a lot of people spent a lot of time talking about what other people think and do, and I'm weary of all that mindreading and proxying.

I would make more sense to speak for ourselves, and ask others to do the same.

I've spent enough time in unproductive committees to know that people have to speak for themselves, point to the parts (specifics about real data and real information), and suggest solutions (else they're just whin'in).

So, the first thing we do with every new offender who wants to say "it was her idea," or "she enjoyed it," or "that 14 year-old can hold her liquor, she wasn't intoxicated," is pneumo-slap 'em and tell 'em to speak only for themselves.

So, there I go ranting about emotional rhetoric, and then I unload with all that about fidelity tests.

-------

Dan,

I get it - but with motorcycles we're talking about philosophical and motivational complications. You known, aesthetics, existence... what makes life pleasurable and what gives us meaning. I gave up rock-climbing after my kid was born, but harbored false hopes that he would enjoy motorcycles as much as I have.

I'll probably put the bike back together, as Mallory said - because its there. Actually, because I can, and because I like fixing things.

I've been talking about selling the bikes for the past year, because I don't ride as much, and showing up to meetings and work with helmet hair isn't cool. So I've been longing for an old porsche, but a del sol or miata would be more sensible. In the meantime I dumped the Subaru Outback on a guy who cut down and ground away the remains 40" diameter stump of a burned up 70 year-old giant silver maple in my front yard. It saved me months of digging, axeing, and chain-sawing, and now I have several tons of wood chips to spread over the sticky clay soil, just in time for the wet season. So I've already been looking for the Porsche project, or perhaps an old BMW Bavaria, but found myself distracted by another shiney-object - an early all-wheel-drive BMW - lots more fun to drive than the Subi and still a tractor in the snow.

Hmmm, a Boss Hoss? Isn't that the bike with the small-block chevy V8?

----------

Have a great Thanksgiving everyone.

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-22-2006 11:51 AM         

Thanks for your thoughtful response. You raise some excellent points. First, let me make clear that I know you weren’t berating anyone. It’s just that after I read my own hastily posted comments to Barry out of context, they seemed too edgy, and I felt bad about it.

Re: Instincts
I must have really screwed up the positioning of what I was trying to say, so I’ll take another shot at it…

• I score strictly by analyzing the charts. Period. But in the “macro” view…
• “Instinctively,” it seems to me that Backster’s complex scoring rules (45 features in all) are better than Utah/DoDPI’s 9-12 (or whatever number that’s in vogue these days). How is having less features better than having more features? To use an analogy, if Backster’s complex scoring rules are the Queen’s English, than Utah/DoDPI rules are Ebonics. Now, if the problem is inter-rater reliability, why must the standards be “dumbed down” to accommodate the raters? Why can’t we “upgrade” the raters?
• “Instinctively,” it seems to me that a specific-issue test with a fear-hope inside track is more illuminating than a test without it.
• “Instinctively,” it seems to me that a marginal Utah test case (like the “dramatic” hypotheticals I described), where an inconclusive case swings to a definitive conclusion by collecting a single additional point in charts 4 and 5 (combined), is less convincing than the decisions reached by the inherently more robust scores used in increasing-threshold formats.

Re: QC review of police/prosecutor tests
Alas, the states in which the potential reviews that I described would have occurred have no licensing laws. The “official police examiner” himself could have been a recent polygraph school grad, who got sick of chasing speeders on the freeway, and, wanting an office job, decided to apply for a vacant polygrapher position and then luckily squeeked through basic poly school with a final average of 70. Or maybe he could be one of those guys who delights in using the polygraph as an electronic rubber hose. Had they gone forward, the actual cases I described would have been reviewed preliminarily by me, and then by Cleve Backster, who would have been a signatory on the QC paperwork. The poly issue would never see a courtroom in ether state.

Re: Blind Scoring
I’d like the luxury of blind scoring 100 cases. How do I get ‘em? For now, all I can cite is my experience to date. When I attended the Backster School, we had a couple of dozen or so take-home cases. With those charts, I never missed a call, and was consistently within a point or two of the “school score.” The final exam at Backster is divided into two parts, written test and chart analysis. I scored a 96 on the chart analysis and, coincidentally, 96 again on the written component. Now, I’m not claiming to have 96% accuracy, but I do claim that while in school I didn’t suffer from the scoring difficulties that *some* of the other students did. During my 30-case field project, my QC reviewers agreed with my findings. I know I blew one case early on – calling a DI inconclusive – because of “operator error” with the EDA component of my LX4000. That was soon corrected. At the Backster work conference last year, we blind scored six or so tests. Again, I was either on the money or within one or two points (never enough to affect the call). Verified cases are hard to come by in my young private practice, but I've had some. The rest? Don't know. This whole discussion reminds me of a Best Practices workshop Don Krapohl conducted in Maine a couple of years ago. He passed out sets of test charts which were scored by the attendees. Odd-numbered exams were to have one outcome (DI or NDI, I forget which) and even-numbered exams the same thing. At the beginning of the workshop, after the attendees scored the charts, there was roughly 50% disagreement. When the exercise was repeated at the end of the Best Practices workshop, there was still substantial disagreement among examiners. I concluded that either the exams Don passed out were purposely walking the tightrope of inconclusivity, or that some examiners simply have a very tough time scoring charts. Perhaps that’s the reason for the rabid fascination with algorithms.

Finally, let me restate a question that has gone unanswered…
If Matte’s Quadri-Track ZCT were satisfactorily validated – with accuracy rates equal to or better than Utah, with the added benefit of diagnosing false positive/negatives – would/should DoDPI (or ASTM for that matter)embrace Matte’s system, incorporate it into its curriculum and advocate its use? If not, why not?

posted 11-22-2006 02:30 PM            

quote:

---

“Instinctively,” it seems to me that Backster’s complex scoring rules (45 features in all) are better than Utah/DoDPI’s 9-12 (or whatever number that’s in vogue these days). How is having less features better than having more features? To use an analogy, if Backster’s complex scoring rules are the Queen’s English, than Utah/DoDPI rules are Ebonics. Now, if the problem is inter-rater reliability, why must the standards be “dumbed down” to accommodate the raters? Why can’t we “upgrade” the raters?

---

I think you answered your own question when you talked about Don's presentation. Not everybody can look at the same data and find what they need to. Anyhow, more complex scoring rules affect reliability (in humans that is - the computer is a different story). That can lead to poorer accuracy, which is what we want to avoid.

quote:

---

“Instinctively,” it seems to me that a specific-issue test with a fear-hope inside track is more illuminating than a test without it.

---

Sure, IF fear is what causes responses, but that's only one theory, and we do know that fear is not necessary for a test to work. What if the OR is responsible? What if it's many variables and changes with each person tested?

quote:

---

“Instinctively,” it seems to me that a marginal Utah test case (like the “dramatic” hypotheticals I described), where an inconclusive case swings to a definitive conclusion by collecting a single additional point in charts 4 and 5 (combined), is less convincing than the decisions reached by the inherently more robust scores used in increasing-threshold formats.

---

Adding charts doesn't increase errors, so your hypothetical isn't consistent with reality. It only reduces inconclusives. The proportions of errors (and non-errors) doesn't change, so this is a straw-man argument.

Don't forget: changing cut-offs - increasing thresholds or otherwise - only changes the types of errors made. They don't eliminate them. What errors are acceptable in any given environment is a policy decision. You and I may not like the decisions some make, but that's not our prerogative.

quote:

---

The “official police examiner” himself could have been a recent polygraph school grad, who got sick of chasing speeders on the freeway, and, wanting an office job, decided to apply for a vacant polygrapher position and then luckily squeeked through basic poly school with a final average of 70.

---

ASTM has QC standards, and this guy wouldn't make the cut, so there's a place to start.

Is a 70 a passing score at most schools? That earned a fat "F" at my school. Below an 80 and you were gone.

quote:

---

Finally, let me restate a question that has gone unanswered…
If Matte’s Quadri-Track ZCT were satisfactorily validated – with accuracy rates equal to or better than Utah, with the added benefit of diagnosing false positive/negatives – would/should DoDPI (or ASTM for that matter)embrace Matte’s system, incorporate it into its curriculum and advocate its use? If not, why not?

---

That's a whole bunch of "ifs," but yes, not only should they, but as Don told you already, they would do so. If all those "ifs" were satisfied, then I believe it would automatically make the ASTM grade.

posted 11-26-2006 08:21 PM            

Not to pick on you, but I wanted to comment on one of your earlier remarks because it shows an error that I have heard on more than one occassion. First, your comment:

Instinctively,” it seems to me that Backster’s complex scoring rules (45 features in all) are better than Utah/DoDPI’s 9-12 (or whatever number that’s in vogue these days). How is having less features better than having more features? To use an analogy, if Backster’s complex scoring rules are the Queen’s English, than Utah/DoDPI rules are Ebonics. Now, if the problem is inter-rater reliability, why must the standards be “dumbed down” to accommodate the raters? Why can’t we “upgrade” the raters?

It is a very common misunderstanding that "more is better" regarding scoring features. Consider an extension of this argument: if you have a system with 45 features and I invent one with 90 features, is mine twice as good as yours? What if I had 900 features, is that even better? The absurdity of this line of thinking should be obvious. There is a limit on the number of diagnostic features that are useful to human evaluators. Right now we know that number to be about 12. It might go up a couple as we explore the issue more, but it is not likely to be a multiple of 12. This is not to defend any particular scoring system, but simply to point out something that you would hear from any psychometrician you ask about human evaluation systems. You will find similarly small numbers of features in other fields that use human evaluators of complex data, such as radiologists. Large numbers of features are a red flag that unreliable features have crept into the list. They might be diagnostic for certain rare individuals, but when applied to the larger population they are either random noise or countraindicative. Some of 45 features you refer to have been disproven in independent research. That's what one would expect if one were familiar with the principles of psychophysiology, signal detection theory, and psychometrics.

I look for a day when polygraph practitioners demand evidence for the claims of the experts. This would be a sign of maturity for the profession, just as psychology moved away from the early introspection approach toward measurement of the phenomena. This led to real progress. I think we're headed that way, too. One sure sign is resistance from experts.

And I would ask for a last favor from my friends here on the Polygraph Place Bulletin Board. Please do not quote my offhand and personal comments on polygraphy as being authoritative. My personal views sans data are no better than anyone else's views sans data. This request is not driven by a false sense of modesty, but rather the practice runs against what I try to impart to others. If anyone wants to cite my research, I'm flattered, but simply quoting "experts" leads to deadend debates between my experts and yours. That does move us forward.

Thanks.

posted 11-27-2006 11:19 PM            

Why can’t we “upgrade” the raters?"

But I ask you to consider a different point of view. I am a Backster grad, so I am familiar with all of that system's rules.

It seems to me that some of the rules are, in fact, really exceptions to a "major" rule.

Example: always score EDA to the lesser CQ, UNLESS the CQ reaction is 4 times that of the RQ reaction (green zone abuse rule).

What that really means is that it doesn't always work to score against a lack-of-reaction and we should sometimes score against the presence-of-reaction.

This exception tells me that the underlying scoring theory may be in error. Any scoring system that needs a large amount of "exceptions" has an inherent problem.

For a really stupid analogy, what if I said " All dogs are friendly in all situations, EXCEPT in these 175 instances"

Maybe we are left thinking that the first statement is in error if there are 175 exceptions to the rule!

My thinking is that a scoring system that works while having fewer rules is probably a better system: certainly one that is less subject to mis-interpretation.

Thanks for listening,